The invention relates to a system and method for testing a distributed, multi-user computer system, and more specifically, relates to the control and testing of such a computer system that is an in-flight entertainment system for commercial aircraft.
Passengers in a commercial aircraft may be provided with personalized entertainment through an in-flight entertainment system (IFES). Such a system typically comprises multiple computer systems connected together with a communication network. Some of the computer systems are dedicated to serving individual passengers (passenger computer), some provide services to all passengers and crew members (server computers), and some (network computers) provide coordination and/or communication functions to allow all the systems to work together. Many of these devices/computers/components are implemented as what are called line-replaceable units (LRUs), i.e. units that are modular in nature, having a similar shape, size, connection, etc. so that they may be rapidly added and removed from the aircraft to minimize downtime. Software to provide the entertainment service is typically distributed among the multiple computers which comprise the IFES.
Overall control of an IFES is performed by members of the aircraft crew, again using computer systems dedicated to that function. These crew computer systems are logically similar to the systems dedicated to each passenger. Both individual passenger computer systems and crew control computer systems have user interface software to accept requests and provide responses. While the possible requests and responses are different for the two types of system users (crew and passengers), the general concept of use is the same.
Testing IFES software can be difficult because of the need to coordinate activities performed using each of the multiple computer systems in the IFES. Previously, most testing was performed manually, by asking a group of individuals to act as the passengers and crew would under various circumstances. The number of individuals in the group might range from one to several dozen, but would almost never reach the number of passengers and crew who might actually be on-board an aircraft using the IFES. Thus, testing did not reliably represent the actual conditions under which the IFES product would be used, particularly with respect to loading and variety of simultaneous operations.
In some cases, testing might be accomplished using a mechanical device to physically press buttons or physically perform other actions that a passenger or crew member might use to control his or her personal experience with the IFES. However, using a mechanical device to simulate all of the passengers and crew members on an aircraft would be very expensive because of the size of the device required. Further, such a device would be unlikely to work properly with different versions of the IFES, requiring multiple expensive devices to test all of the IFES products available to customers. And because of the complexity of such a device, it would likely be difficult to separate issues related to the device from issues related to the IFES under test. Use of a mechanical device is thus not an implemented technique in the IFE industry.
Manual testing is extremely difficult to coordinate, and the collection of results is haphazard at best. It is nearly impossible to get a group of individuals to perform actions in a synchronized manner except with extensive practice. Even then, the repertoire of practiced actions would be much smaller than the set of possible actions to be tested. Assembling a large group of testing individuals is a significant expense and the members of the group are unlikely to be the same for multiple tests. The result is that such tests cannot be reproduced reliably, and therefore it is generally impossible to verify that a previously identified problem has been fixed.
Although it may be possible to add mechanical or electronic test connections to the computer systems which comprise an IFES, and also possible to conduct manual testing using non-professional individuals to save some cost, these testing techniques are prohibitively expensive in both labor and materials.
Adding standard mechanical or electronic test connections to computer systems increases their cost, reducing the ability of such systems to compete successfully in the market. Such connections would also increase the complexity of such systems, increasing the number of possible failure points, and reducing the reliability. They might also increase the weight or necessary power, compounding competitive strain.
Because full system testing requires the physical presence of all the computer systems which comprise an IFES, and because such systems are expensive and in short supply, it is not practical to conduct such tests using ad-hoc groups of individuals at physically diverse locations. Individuals performing the tests must have substantial familiarity with the IFES to avoid wasted time and may require a degree of (non-governmental) security clearance to be permitted access to the test environment.